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DEE  H.  ANDREWS1 

1.  STORY  TYPES  AND  THE  HERO  STORY 


For  thousands  of  years  societies  have  taught  key  principles  through  storytelling 
(Brady,  1997;  MacDonald,  1998).  In  some  cultures  without  a  written  language 
storytelling  was  the  only  way  to  convey  a  society’s  culture,  values,  and  history. 
Instructional  tools  have  been  used  by  great  teachers  and  leaders  in  the  forms  of 
parables,  legends,  myths,  fables,  and  real-life  examples  to  convey  important 
instruction  (Benedict,  1934;  Brown  &  Duguid,  1998;  Davenport  &  Prusak,  1998; 
Leonard-Barton,  1995).  Fictional  and  non-fictional  examples  have  always  been 
powerful  teaching  tools.  Storytelling  as  instruction  is  still  heavily  used  today.  The 
military,  aviation,  medical,  law,  and  business  communities  are  just  a  few  groups 
which  rely  heavily  on  storytelling  as  methods  for  teaching  key  principles  of  their 
discipline  and  to  help  build  analytical  prowess  in  students  and  trainees. 

While  many  definitions  of  “story”  can  be  found  in  the  literature,  this  author  is 
partial  to  two  of  them.  Labov  (1972)  defines  a  story  or  a  narrative  “as  one  method 
of  recapitulating  past  experiences  by  matching  a  verbal  sequence  of  clauses  to  the 
sequence  of  events”  (p.  359-60)  and  at  a  minimum  a  “sequence  of  two  clauses 
which  are  temporally  ordered”  (p.  360).  Denning  (2009)  states  that,  “A  narrative  or 
story  in  its  broadest  sense  is  anything  told  or  recounted;  more  narrowly,  and  more 
usually,  something  told  or  recounted  in  the  form  of  a  causally-linked  set  of  events; 
account;  ta!e,:[sic]  the  telling  of  a  happening  or  connected  series  of  happenings, 
whether  true  or  fictitious.” 

There  are  many  publications  that  give  guidance  about  how  best  to  formulate  and 
use  stories  for  use  in  instruction.  Many  of  these  offer  prescriptive  guidelines  to 
those  who  teach  using  storytelling.  Examples  include:  Gershon  and  Page  (2001), 
Harries,  C.  (2003),  Hill,  Gordon,  and  Kim  (2004),  Merrill,  (2002),  Preczewski, 
Hughes-Caplow,  and  Donaldson,  (1996).  However,  there  is  not  a  large  theoretical 
foundation  or  empirical  evidence  about  the  storytelling  technique.  As  have  teachers 
and  instructors  for  thousands  of  years,  we  know  that  storytelling  is  a  very  effective 
instructional  method.  However,  the  key  questions  are,  “Why  do  stories  work  so 
well  in  instruction?  What  are  the  features  and  characteristics  of  stories  that  make 
them  work?  How  can  stories  be  improved  for  instruction?” 

This  book  stems  from  a  workshop  organized  by  the  U.S.  Air  Force  Research 
Laboratory2.  The  military  is  interested  in  better  instructional  storytelling  because 
military  instructors  have  historically  relied  heavily  on  that  technique.  Whether  the 
instruction  is  done  from  the  platform,  through  texts,  via  computer-based 
instruction,  in  simulators,  or  in  the  field,  stories  are  told.  In  fact,  storytelling  does 
not  stop  in  the  classroom  or  in  a  formal  training  setting.  Much  of  the  culture  and 
tradition  of  the  military  is  passed  along  in  stories  as  military  personnel  stand  watch. 
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socialize  after  hours,  and  interact  while  traveling  to  and  from  missions  and 
exercises.  In  most  cases  these  instructional  stories  stem  from  actual  experience  in 
combat  operations  or  in  training  exercises. 

They  may  be  “there  I  was,  in  danger”  stories  where  individual  instructors  tell 
stories  from  their  personal  experience,  or  they  may  be  scenarios  for  simulators  or 
field  exercises  that  come  from  third  person  accounts  of  battle,  but  stories  have 
proven  for  thousands  of  years  to  have  a  positive  instructional  effect.  Hence  the 
military  interest  in  better  instructional  storytelling.  One  of  the  workshop’s  major 
goals  was  to  explore  different  ways  to  produce  more  instructionally  effective  stories. 

Over  the  course  of  two  days  at  the  workshop,  a  variety  of  presentations  were 
given  exploring  four  techniques  using  stories  in  instruction;  case-based,  problem- 
based,  scenario-based,  and  narrative-based  methods  of  instruction.  The  group 
examined  and/or  developed  instructionally  relevant  definitions  for  the  different 
types  of  storytelling  approaches.  Another  goal  of  the  workshop  was  for  attendees  to 
develop  and  discuss  key  research  questions  related  to  the  theme  of  the  workshop. 

The  first  section  of  the  chapter  describes  the  four  types  of  stories  considered  by 
workshop  participants.  What  are  their  definitions  and  how  are  they  used  in 
instruction?  The  chapter  then  examines  some  of  the  research  questions  that 
emerged  from  the  workshop.  A  major  goal  was  to  develop  a  set  of  questions  that 
might  lead  to  the  development  of  a  more  empirical  foundation  for  instructional 
storytelling  than  currently  exists.  Theories  help  explain  and  predict  phenomena. 
As  the  workshop  did  not  identify  a  strong  theoretical  base  for  instructional 
storytelling,  the  chapter  is  concluded  by  briefly  examining  the  ideas  of  scholar, 
Joseph  Campbell,  who  spent  his  professional  career  theorizing  about  the  place  of 
stories  (myths)  in  cultures.  He  believed  that  mythical  stories  are  much  more  than 
just  entertainment,  but  are  actually  a  deeply  engrained  part  of  our  psyches  that  are 
often  used  as  learning  tools.  It  is  but  one  example  of  what  might  provide  a  theory 
foundation  for  storytelling  as  a  method  of  instruction. 

FOUR  STORY  TYPES 

While  all  four  main  storytelling  instructional  methods  (case-based  instruction, 
problem-based  instruction,  scenario-based  instruction,  and  narrative-based  instruction) 
share  a  common  element  —  stories  -  the  four  do  have  differences  in  definition, 
purpose,  use  of  the  story,  and  outcomes.  Each  method  presents  learners  with  a 
temporally  ordered  sequence  of  information  and  employs  an  attention-focusing 
mechanism.  Uniting  these  methods  through  a  common  characteristic  enables 
researchers  to  draw  on  one  another’s  work  for  insights  into  the  learning  process. 
Andrews,  Hull,  and  Donahue  (2009)  describe  these  story  types  in  greater  detail  and 
provide  concrete  examples.  This  book  is  organized  around  these  four  story  types. 

Case-Based  Instruction 

Cases  are  stories  that  have  occurred  in  the  past.  They  are  widely  used  in  contexts 
such  as  medical,  law,  and  business  schools.  Case-based  instruction  fixes  the 
problem  and  solution,  but  the  learner  is  placed  outside  the  story  context  (Bames, 
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Christensen,  &  Hansen,  1 994).  The  learner  must  discover  the  key  facts  and  events 
as  they  occurred;  hence  case  studies  have  a  historical  nature.  Because  they  are 
historical,  cases  do  not  allow  a  learner  to  alter  their  outcome  or  processes. 
Rather,  the  student  must  apply  critical  thinking  and  theories  to  the  existing  facts 
to  be  able  to  form  hypotheses  about  why  the  facts  of  the  case  occurred  as  they 
did.  A  major  advantage  of  cases  when  compared  to  the  other  three  types  of 
stories  is  that  they  are  imbued  with  authority  which  comes  from  the  actual  facts 
of  the  stories  (Abbot,  1 992). 


Narrative-Based  Instruction 

Narrative-based  instruction  fixes  the  problem,  the  solution,  and  the  learner  all  within 
the  context  that  the  story  frames  (Cobley,  2001).  The  storyteller  or  narrator  controls 
all  of  the  information  received  by  the  learner.  Narratives  can  be  either  fictional  or 
non-fictional.  They  seek  to  emotionally  immerse  the  learner  in  the  narrative’s 
situation;  probably  more  deeply  than  any  of  the  other  three  story  types.  For  this 
reason,  narratives  often  are  told  for  entertainment’s  sake,  often  without  pursuing  an 
instructional  objective.  A  narrative  seeks  to  express  a  series  of  events;  however  it 
does  not  necessarily  have  to  tell  the  events  in  a  chronological  sequence. 


Scenario-Based  Instruction 

Scenarios  state  fixed  solution  criteria,  but  not  necessarily  fixed  solutions.  The 
learner  is  positioned  in  a  place  that  allows  them  to  interact  with  the  scenario  and 
produce  different  outcomes  depending  on  their  decisions  and  actions.  They  can  be 
fictional  or  non-fictional.  However,  for  purposes  of  instruction  they  often  come 
from  history  (Salas,  Wilson,  Priest,  &  Guthrie,  2006).  Scenarios  are  heavily  used  in 
operational  training  such  as  the  nuclear  power  industry  and  the  military  because 
they  require  active  interaction  by  the  learner  and  can  be  given  operational 
characteristics.  While  many  scenarios  are  drawn  from  actual  cases,  they  can  be 
altered  (sometimes  significantly)  to  suit  the  purpose  of  instruction  and  evaluation 
measurement.  This  ability  to  accurately  measure  learner  responses  in  a  scenario- 
driven  simulation,  simulator,  or  instructional  game  makes  scenarios  effective 
places  to  try  out  new  theories,  approaches,  and  procedures  for  solving  operational 
problems.  Learners  can  gain  valuable  lessons  from  the  experience.  The  main  goal 
of  scenario-driven  instruction  is  to  improve  performance. 

Problem-Based  Instruction 

The  final  story  type  is  especially  suited  for  teaching  learners  about  how  to  best 
solve  ill  structured  problems  that  do  not  have  optimal  solution  criteria  or 
parameters  (Hmelo-Silver,  2004;  Savery,  2006).  Problem-based  instruction 
requires,  or  at  least  allows,  the  learner  to  take  charge  of  their  own  learning 
process  and  activities.  This  uses  the  problem  (fictional  or  non-fictional)  as  a 
mechanism  for  conveying  knowledge  to  the  learner.  The  learning  is  usually  done 
in  a  team  setting,  where  each  team  member  must  provide  collaborative  help  in 
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finding  a  solution  (Boud  &  Feletti,  1997).  A  key  is  that  while  a  teacher  might  help 
the  learning,  each  learner  and  the  learning  team  must  take  responsibility  for  defining 
a  path  to  solving  the  problem  and  then  applying  facts  and  skills  to  reach  a  solution 
(Savery,  1998). 

KEY  INSTRUCTIONAL  STORY  RESEARCH  QUESTIONS 

Regardless  of  the  type  of  story,  there  are  many  research  questions  about  instructional 
storytelling  whose  answers  might  help  in  developing  more  instructionally  effective 
stories.  Following  are  a  few  examples  of  research  questions  that  resulted  from  the  Air 
Force  Research  Laboratory  workshop: 

-  How  much  impact  does  the  individual  storyteller  have  on  the  instructional 
effectiveness  of  the  story,  and  more  importantly,  what  about  the  storyteller 
makes  him  or  her  effective?  Can  instructors  be  taught  how  to  be  better 
storytellers,  or  are  some  people  just  born  storytellers  and  others  are  not? 

-  How  long  are  lessons  learned  from  stories  retained  by  the  learner  versus 
learning  via  other  methods?  All  of  us  can  likely  remember  instructional  stories 
from  our  early  childhood,  but  are  there  characteristics  of  some  stories  that  lead 
to  longer  retention  than  stories  lacking  those  characteristics? 

-  Do  different  types  of  stories  have  different  effects  on  different  learning  styles? 
Stated  differently,  do  certain  learning  styles  respond  more  effectively  to 
different  story  types? 

-  Are  there  material  differences  in  the  genesis,  form,  and  effectiveness  of  case- 
based  vs.  scenario-based  vs.  problem-based  vs.  narrative-based  instruction?  That 
is,  are  these  just  different  names  for  the  same  method  or  are  they  really  different 
in  some  important  ways?  If  so,  does  it  matter  to  the  learner?  Continuing  along 
that  same  line  of  questioning,  do  different  storytelling  techniques  have  differential 
effects  on  different  learners? 

-  Why  do  stories  work  from  a  cognitive  standpoint?  It  seems  they  certainly  have 
associative  properties  that  can  make  the  learning  relevant  to  the  user,  but  can 
that  association  be  empirically  analyzed  and  modeled  so  that  more  effective 
stories  can  be  developed? 

-  Is  it  better  to  (a)  first  present  examples  via  storytelling  and  then  extract  a  general 
principle  from  examples,  or  (b)  first  learn  a  general  principle  and  then  listen  to 
examples  via  storytelling?  What  does  theory  and  evidence  tells  us  about  this 
question  that  could  be  applied  to  improving  storytelling  as  instruction? 

CAMPBELL  AND  THE  HERO  STORY 

The  Air  Force  Research  Laboratory  workshop  examined  the  questions,  “Is  there  a 
theory  of  storytelling  as  instruction?  If  not,  should  there  be,  and  can  there  be?” 
Those  questions  were  addressed  but  not  really  answered  in  the  workshop,  largely 
because  of  the  brief  time  the  participants  had  to  discuss  the  issue.  One  direction 
to  turn  for  a  possible  theory  might  be  to  the  ideas  of  a  thinker  who  believed  that 
stories  have  a  special  place  and  purpose  in  every  culture  around  the  world, 
including  for  pedagogical  purposes.  The  mythologist  Joseph  Campbell  spent  his 
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career  exploring  myths  from  many  cultures  (Campbell,  2008;  Campbell  &  Moyers, 
1991).  He  concluded  that  virtually  all  myths  have  similar  structures  that  revolve 
around  a  hero.  A  writer,  Christopher  Vogler,  (2009)  provides  a  brief  overview  of 
Campbell’s  findings  about  the  hero  myth. 

The  hero  is  introduced  in  his  ordinary  world,  where  he  receives  the  call  to 
adventure.  He  is  reluctant  at  first  but  is  encouraged  by  the  wise  old  man  or 
woman  to  cross  the  first  threshold,  where  he  encounters  tests  and  helpers.  He 
reaches  the  innermost  cave,  where  he  endures  the  supreme  ordeal.  He  seizes 
the  sword  or  the  treasure  and  is  pursued  on  the  road  back  to  his  world.  He  is 
resurrected  and  transformed  by  his  experiences.  He  returns  to  his  ordinary 
world  with  a  treasure,  boon,  or  elixir  to  benefit  his  world. 

Campbell  posits  the  existence  of  a  Monomyth  (Joyce,  1995),  which  is  a  clearly 
defined  pattern  that  seems  to  fit  every  well  known  myth  from  every  culture.  He 
believed  that  humans  resonate  with  the  themes  and  the  imagery  of  myths  because 
the  stories  are  a  metaphor  for  life.  That  is  why  they  have  such  a  powerful  educational 
value.  Campbell  believed  that  myths  are  psychologically  “true”.  Even  when  the 
myths  portray  fantastic  events  and  creatures,  we  still  respond  to  them  because  they 
map  to  our  psyches. 

Campbell  drew  inspiration  from  the  ideas  of  noted  psychiatrist  and  critical 
thinker,  Carl  Jung.  Jung  developed  the  concept  of  the  “Archetypes”  (Jung,  1981). 
He  believed  that  there  are  characters  which  repeatedly  populate  the  dreams  and 
myths  of  all  mankind  across  cultures.  He  postulated  that  the  mind  is  reflected  in 
these  Archetypes.  He  believed  that  myths  and  stories  map  to  the  archetypes  of  our 
minds  and  that  is  why  they  have  such  power  in  every  culture.  They  tap  into  a 
collective  unconsciousness  and  map  on  to  our  psyches. 

Campbell’s  generalized  hero  myth  concept  is  not  without  its  critics.  Some 
scholars  who  study  myths  believe  his  twelve  stages  are  too  formulaic,  and  their  use 
squeezes  from  myths  the  true  nature  of  what  makes  cultures  different. 

Campbell  (1990)  believed  that  mythology  had  four  major  functions;  mystical, 
cosmological,  sociological,  and  pedagogical  and  that  myths  and  stories  have 
significant  contributions  to  learning  in  all  cultures.  It  is  because  of  his  interest  in 
the  pedagogical  nature  of  myths  that  we  find  interest  in  the  possible  use  of  his  ideas 
for  laying  a  theoretical  foundation  for  research  of  instructional  storytelling. 

Campbell  believed  that  the  hero  myth  helps  us  cope  with,  if  not  answer,  the  key 
universal  questions  humans  have  about  the  universe  and  their  existence.  Questions 
such  as,  “Where  have  we  come  from?”,  “Why  are  we  here?”,  and  “Where  are  we 
going?”  The  myths  can  be  real  or  fictional  and  still  have  the  same  powerful  teaching 
influence.  He  believed  that  these  stories  are  ways  for  entire  cultures  to  express  their 
identity  and  answer  questions  about  their  beliefs.  As  long  as  the  pattern  of  the  hero 
myth  is  adhered  to  these  myths  and  stories  can  teach  for  many  generations. 

Campbell  identified  twelve  stages  of  the  hero  myth: 

1 .  Hero  introduced  to  his  ordinary  world 

2.  Call  to  adventure 

3.  Reluctant  hero 
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4.  Hero  encouraged  by  wise  old  man 

5.  Hero  passes  first  threshold 

6.  Hero  encounters  tests  and  helpers 

7.  Hero  reaches  perilous  place 

8.  Hero  undergoes  serious  test 

9.  Hero  takes  control  of  the  prize 

10.  Hero  makes  final  escape 

1 1 .  Hero  is  transformed  by  the  quest 

1 2.  Hero  uses  prize  to  benefit  mankind 

Many  military  training  instructional  stories  follow  Campbell’s  twelve  step  model. 
The  instructor  recounts  a  tale,  either  first  or  third  hand,  about  a  military  member  or 
team  (hero)  that  has  a  mission  to  complete  (quest  or  prize).  Along  the  way,  many 
obstacles  are  encountered.  These  obstacles  may  be  the  enemy,  the  weather  or 
failing  equipment.  The  hero  overcomes  the  obstacles,  accomplishes  the  mission, 
and  benefits  the  larger  military  mission,  which  will  benefit  a  particular  group  (the 
hero’s  military  forces  in  the  field  or  nation,  and  eventually  the  entire  world). 

Perhaps  the  educational  influence  of  myths  and  stories  will  have  a  stronger 
emotional  influence  if  they  use  the  steps  as  a  pattern.  Campbell’s  ideas  could 
provide  theoretical  underpinning  for  developing  better  instructional  stories.  If  his 
theoretical  twelve-step  model  is  correct,  then  perhaps  we  should  start  with  that 
model  to  construct  many  of  our  instructional  stories.  Research  in  this  area  could 
offer  evidence  about  the  efficacy  of  Campbell’s  model  as  it  applies  to  instructional 
stories.  While  his  ideas  are  but  one  explanation  for  the  power  of  instructional 
stories,  his  views  have  received  considerable  attention  from  both  the  scholarly 
world  and  the  popular  press.  Campbell’s  concepts  can  help  researchers  as  they 
explore  the  storytelling  questions  described  in  the  previous  section. 

We  ask  the  reader  to  consider  other  possible  foundational  theories  for 
instructional  storytelling  as  they  read  this  book.  It  is  vital  that  instructional 
storytelling  be  founded  on  well  developed  theories  so  that  it  can  take  its  rightful 
place  alongside  other  proven  instructional  methods  that  have  strong  theoretical 
bases. 


CONCLUSION 

Stories  used  for  instruction  have  a  history  as  long  as  the  spoken  word.  They  will 
continue  regardless  of  whether  theory  bases  for  instructional  storytelling  are 
constructed  or  not.  In  like  manner,  the  use  of  instructional  stories  will  remain  a  part 
of  instruction  regardless  of  whether  empirical  research  is  focused  on  the  domain. 
What  theory  building  and  empirical  research  should  contribute  is  a  better 
understanding  of  how  to  best  construct  and  use  such  stories. 

The  four  story  types  are  not  mutually  exclusive.  Their  construction  and  use 
overlap.  However,  their  differences  are  important  enough  to  treat  separately. 
Empirical  research  might  help  build  a  prescriptive  guide  for  when  and  how  each 
type  can  be  optimally  used  in  education.  Perhaps  Campbell’s  ideas  are  correct 
and  all  humans  have  an  innate  understanding  of  the  power  of  stories.  Research 
might  also  help  us  understand  when  and  how  to  use  the  different  story  types  in 
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combination.  If  so,  research  can  help  the  instructional  process  by  proposing 
underlying  story  theories  that  help  explain  and  predict  their  optimal  use  for 
instruction. 


NOTES 

1  The  opinions  expressed  in  this  chapter  are  those  of  the  author  and  do  not  necessarily  represent  the 
official  views  or  policies  of  the  US  Department  of  Defense. 

1  Storytelling  as  an  Instructional  Method  Workshop:  In  search  of  Theoretical  and  Empirical 
Foundations,  November  7  -  8,  2006,  Mesa,  AZ.  United  States  Air  Force  Research  Laboratory,  71  l'h 
Human  Performance  Wing,  Human  Effectiveness  Directorate,  Warfighter  Readiness  Research  Division 
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3.  STORYTELLING,  ARCHETYPES  AND  SYSTEM 
DYNAMIC  MODELING 


As  discussed  by  Andrews,  Hull  and  Donahue  (2009),  storytelling  has  been  used  for 
thousands  of  years  as  a  means  for  teaching  members  of  various  societies  about  their 
culture,  values,  and  history.  Indeed,  it  would  be  difficult  to  overstate  the  importance 
of  storytelling  as  a  universal  method  of  teaching.  Moreover,  in  many  cases, 
storytelling  is  effective  by  its  use  of  archetypes.  An  archetype  represents  an  original 
pattern,  a  prototype,  or  ideal  example;  archetypes  represent  universal  symbols.  For 
example,  one  archetype  would  be  that  of  a  ‘hero’.  The  archetype  of  a  hero  would 
evoke  a  number  of  images  and  expectations  on  the  part  of  the  listener  or  reader,  such 
as  bold  and  fearless,  which  would  aid  in  understanding  the  meaning  of  the 
interactions  of  that  individual  with  others  in  the  story.  In  this  way,  listeners  or  readers 
of  a  story  can  easily  relate  to,  and  understand,  the  meaning  conveyed  by  the  evolving 
pattern  of  actors  and  events  by  understanding  the  meaning  of  the  archetypes. 

ANALOGICAL  REASONING  AND  STORYTELLING 

The  use  of  archetypes  in  storytelling  is  an  effective  means  by  which  to  communicate 
essential  information  because  archetypes  involve  reasoning  by  analogy.  As  discussed 
by  Holy  oak,  Gentner  and  Kokinov  (2001)  and  Juthe  (2005),  analogical  reasoning 
involves  making  inferences  from  the  similarity  of  relationships  of  elements  across 
two  or  more  domains.  These  authors  note  that  by  dealing  with  the  shared  relational 
patterns  among  potentially  diverse  objects,  analogical  reasoning  involves  dynamic, 
context-sensitive  mental  representations.  Analogy  can  involve  specific  cases  or 
highly  abstract  concepts  or  schemas.  Reasoning  by  analogy  may  also  be  referred  to 
as  case-based  reasoning,  wherein  the  structures  and  relations  of  situations  stored  in 
memory  are  applied  to  novel  problems. 

Analogical  reasoning  involves  creating  a  mental  structural-alignment  process,  or 
mapping,  of  relationships  from  one  domain  to  another  in  order  to  transfer 
inferences  between  domains  (see  Holy  oak,  Gentner  &  Kokinov,  2001).  Hofstadter 
(2001)  makes  a  strong  case  that  analogical  processing  is  at  the  very  core  of 
cognition.  He  notes  that  the  development  of  concepts,  which  are  employed  in  many 
mental  functions  such  as  reasoning  and  thinking,  requires  individuals  to  mentally 
chunk  together  the  core  relational  properties  of  objects  and  situations,  using 
analogical  abilities,  into  a  concept.  Indeed,  humans  typically  perform  very  well 
using  analogies  for  reasoning,  especially  when  the  connection  between  the  analogue 
and  target  is  made  explicit  (Gick  &  Holyoak,  1980,  1983).  Analogical  reasoning 
appears  to  be  one  of  the  more  fundamental  thinking  skills  humans  possess. 


D.  H.  Andrews,  T.  D.  Hull  and  K.  DeMeester  (eds.).  Storytelling  as  an  Instructional  Method: 
Research  Perspectives,  29-34. 

©  2010  Sense  Publishers.  AH  Rights  Reserved. 


PATTERSON 


Juthe  distinguishes  between  same-domain  analogies  wherein  the  elements  of  the 
analogue  and  target  are  drawn  from  the  same  domain,  and  different-domain 
analogies  wherein  the  elements  of  the  analogue  and  target  are  drawn  from  different 
domains  (the  latter  case  involves  metaphors  or  parables).  Same-domain  analogies 
are  likely  to  yield  more  specific  courses  of  action  than  different-domain  analogies 
(Markman  &  Moreau,  2001). 

Although  analogical  reasoning  can  be  classified  as  a  different  form  of  reasoning 
from  deductive  and  inductive  reasoning  (Juthe,  2005),  Holland,  Holyoak,  Nisbett  and 
Thagard  (1986)  suggest  that  analogical  reasoning  is  a  form  of  inductive  reasoning 
because  analogy  may  entail  the  integration  of  information  from  a  number  of  diverse 
sources  or  analogues.  These  authors  discuss  how  analogical  reasoning  entails  creating  a 
mental  model  of  the  target  problem  by  ‘modeling’  the  model  (i.e.,  relations)  contained 
in  the  analogue  problem,  which  creates  a  new  model  that  can  be  applied  to  the  target 
problem — therefore  analogical  reasoning  involves  a  second-order  morphism. 

Moreover,  Hofstadter  (2001)  and  Chalmers,  French  and  Hofstadter  (1992)  argue 
that  analogical  reasoning  is  intimately  involved  in  high-level  perception.  According 
to  this  view,  raw  sensory  information  is  taken  in  by  sensory  systems  which,  at  the 
highest  level,  evoke  perceptual  processes  relating  to  mental  categories  and  concepts. 
Here,  the  meaning  of  the  sensory  information  is  interpreted  via  an  application  of 
structural  alignment  and  mapping  from  stored  representations  in  memory.  Thus,  the 
elicitation  of  stored  mental  categories  and  concepts  by  sensory  input  is  analogical 
processing,  a  form  of  high-level  pattern  perception. 

Thus,  storytelling,  and  its  use  of  archetypes,  is  an  effective  method  for  teaching 
and  conveying  information  because  analogical  reasoning,  in  the  context  of 
storytelling,  would  involve  the  creation  of  dynamic,  context-sensitive  mental 
schemas  of  relational  patterns  conveyed  in  a  story.  An  individual  could  then  transfer 
these  schemas  to  their  real-life  circumstances  which  would  assist  the  individual  in  the 
interpretation  of  previous,  present,  or  future  real-world  events.  This,  in  turn,  could 
facilitate  problem  solving. 

It  is  interesting  to  note  that  storytelling  is  a  form  of  mental  simulation,  entailing 
a  high  degree  of  imagery.  Moreover,  analogical  reasoning  would  also  be  involved  in 
virtually  all  other  forms  of  simulation  as  well.  In  these  cases,  a  mapping  of  relational 
schemas  and  transfer  of  meaning  and  inferences  would  occur  from  the  simulation 
context  to  the  real-world.  To  state  it  another  way,  the  importance  of  simulation  in 
training  and  other  endeavors  is  to  be  found  in  the  analogical  reasoning,  and  pattern 
recognition,  that  is  brought  into  play  by  the  simulation  context. 

Storytelling  is  a  relatively  informal  method  for  conveying  information  by 
simulation.  On  the  other  end  of  the  continuum  are  formal  methods  (e.g., 
mathematical)  for  conveying  information  via  simulation.  In  the  following  section, 
we  discuss  one  such  method,  namely  system  dynamics  modeling. 

SYSTEM  DYNAMICS  MODELING 

System  dynamics  modeling  (Forrester,  1961,  1968;  Sterman,  2000)  is  a  systematic 
framework  for  analyzing  complex  systems  by  simulating  them  as  interconnected 
multiple  positive  and  negative  feedback  loops.  Such  complex  systems  are  set  up  as 
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systems  of  differential  equations  that  are  solved  step-wise  using  numerical  integration 
techniques.  In  this  context,  the  behavior  of  a  system  is  given  by  the  interconnectiveness 
of  the  system  elements — not  by  the  elements  themselves.  The  advantage  of  this 
approach  is  that  it  permits  a  precise  specification  of  the  variables,  and  their 
interrelationships,  in  the  problem  space.  Because  this  approach  entails  the  creation  of 
dynamic  models  of  relational  patterns  whose  simulation  can  generate  inferences  that 
can  be  transferred  to  real-life  circumstances,  system  dynamics  represents  an  approach 
that  is  analogous  to  storytelling  in  its  implicit  use  of  analogical  reasoning  as  a  means  for 
learning  and  teaching.  Some  of  the  computational  structures  created  in  system 
dynamics  modeling  can  be  taken  to  be  a  form  of  an  archetype.  The  only  difference  is 
that  system  dynamics  is  a  more  formal  approach  for  framing  the  analytical  reasoning. 

In  system  dynamics  modeling,  complex  systems  are  analyzed  by  using  a  small  set 
of  elements  which  are  interconnected  to  create  a  system.  These  elements  include  an 
object  called  a  ‘stock’,  which  is  a  rectangular  structure  representing  the  mathematical 
process  of  integration;  a  “flow”,  which  is  a  thick  arrow  representing  rates  of  change 
or  derivatives;  a  connector,  which  is  a  thin  arrow  representing  a  feedback  connection 
as  well  as  other  types  of  connections;  and  a  ‘converter’,  which  is  a  circle  representing 
an  expression,  symbol,  or  conditional  logic.  These  elements  are  used  to  create  and 
model  positive  feedback  loops  and  negative  feedback  loops,  which  are  archetypes. 

Positive  feedback  refers  to  a  deviation-amplifying  process  wherein  a  given  entity 
grows  in  a  fixed  proportion  to  its  size  over  time,  creating  exponential  growth.  This 
type  of  feedback  is  considered  destabilizing.  An  example  of  positive  feedback  is  the 
initial  growth  of  cells  in  a  Petri  dish.  In  this  case,  a  small  amount  of  cells  at  the 
beginning  of  the  process  grows  over  time  to  become  very  large  as  more  and  more 
cells  create  more  cells.  Or  consider  the  example  of  the  adoption  of  a  technological 
innovation  by  a  hypothetical  society.  Here,  a  small  amount  of  innovation  adoption  at 
the  beginning  of  the  process  grows  over  time  to  become  very  large  as  more  and  more 
individuals  spread  the  innovation  by  word  of  mouth.  In  system  dynamics  terms,  the 
rate  of  change  in  system  state  is  directly  proportional  to  the  state  of  the  system  at  any 
point  in  time  (called  first-order  positive  feedback): 

dS/dt  =  kS  (1) 

where  S  =  the  entity  or  state  of  the  system,  k  =  the  proportionality  or  fractional 
growth  rate,  and  dS/dt  is  the  rate  of  change  over  time.  In  a  system  dynamics  block 
diagram,  positive  feedback  can  look  like  the  following: 
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Repetitively  over  time,  the  amount  of  the  system  state  is  fed  back  to  provide 
input  so  that  a  portion  of  it  is  added  to  the  accumulation,  which  creates  exponential 
growth  over  time.  This  is  one  kind  of  archetype  in  system  dynamics  modeling. 

Negative  feedback,  on  the  other  hand,  refers  to  a  goal-seeking  process  wherein 
a  given  entity  or  state  of  the  system  is  adjusted  over  time  to  match  a  given  goal 
(there  are  two  versions  of  this  concept;  only  one  is  discussed  below).  This  type  of 
feedback  loop  involves  the  adjustment  of  a  system  in  the  context  of  intentions;  it 
is  considered  stabilizing.  An  example  of  negative  feedback  is  the  thermostat  in  a 
room  which  controls  the  temperature.  In  this  case,  the  room  is  cooled  if  the 
temperature  is  higher  than  the  setting  (goal),  and  the  room  is  heated  if  the 
temperature  is  lower  than  the  setting.  Or  consider  the  cruise  control  on  an 
automobile  whose  adjustments  attempt  to  keep  the  speed  of  travel  equal  to  the 
setting  on  the  control.  These  types  of  systems  are  termed  controllers  or 
regulators.  In  system  dynamics  terms,  the  rate  of  change  in  system  state  is 
directly  proportional  to  the  difference  between  the  goal  and  the  state  of  the 
system  at  any  point  in  time  (called  first-order  negative  feedback); 

dS/dt  =  k(G  -  S)  (2) 

where  G  =  goal,  S  =  entity  or  state  of  the  system,  k  =  the  proportionality  or 
fractional  change  rate,  and  dS/dt  is  the  rate  of  change  over  time.  In  a  system 
dynamics  block  diagram,  negative  feedback  looks  like  the  following: 


Note  that  when  the  magnitude  of  S,  the  state  of  the  system,  is  less  than  that  of 
the  goal  (G),  the  expression  k(G-S)  is  positive  and  the  action  taken  is  to  add  a 
fractional  amount  to  S;  and  when  the  magnitude  of  S  is  greater  than  that  of  the  goal 
(G),  the  expression  k(G-S)  is  negative  and  the  action  taken  is  to  subtract  a  fractional 
amount  from  S.  These  actions  create  exponential  growth  or  decay,  respectively, 
toward  the  goal  state  over  time.  This  is  another  kind  of  archetype  in  system 
dynamics  modeling. 

These  two  types  of  feedback  loops,  positive  and  negative,  can  be  combined  in 
different  ways  to  model  different  systems.  For  example,  positive  and  negative 
feedback  can  be  coupled  to  create  a  logistic  function,  which  looks  like  an  “S”- 
shaped  curve  and  is  a  common  expression  for  capacitated  growth,  such  as  the 
growth  of  cells  in  a  Petri  dish  beyond  the  initial  growth  stage.  The  initial  growth  is 


32 


STORYTELLING,  ARCHETYPES  AND  SYSTEM 


approximately  exponential,  but  as  more  cells  grow,  the  available  population  shrinks 
as  accumulation  approaches  capacity,  and  growth  asymptotes.  The  differential 
equation  for  this  process  is: 


dP/dt  =  (kP)  -  (kP)(P/C)  (3) 

where  P  =  a  hypothetical  population,  k  =  fractional  growth  or  decay  factor,  and  C  = 
capacity.  In  a  system  dynamics  block  diagram,  this  expression  looks  like  the 
following: 

cIP/dt  -  kP  -  kP  (P/C) 


This  expression  is  modeled  as  a  pair  of  coupled  positive  and  negative  feedback 
loops,  with  kP  defining  the  positive  loop,  and  -(kP)(P/C)  defining  the  negative 
loop.  In  the  latter  expression,  the  effect  of  the  negative  loop  -kP  (exponential 
decay)  is  weighted  by  the  factor  P/C,  so  that  when  it  is  early  in  the  simulation  and 
P  is  small,  then  P/C  is  small,  and  the  positive  growth  loop  dominates  the  process, 
which  gives  a  high  rate  of  overall  growth.  Later  in  the  simulation  P  becomes  large 
and  near  capacity,  P/C  is  near  1 .0,  and  the  negative  loop  begins  to  balance  out  the 
positive  loop,  and  growth  asymptotes-the  system  reaches  dynamic  equilibrium. 
The  figure  below  depicts  the  output  of  this  model,  which  is  a  logistic  function: 


Figure  1.  Interaction  between  two  archetypes. 
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Thus,  this  logistic  function,  which  is  a  function  of  capacitated  growth,  is  the 
result  of  the  interaction  between  two  archetypes,  a  positive  feedback  loop  and  a 
negative  feedback  loop. 

The  two  archetypes  of  positive  and  negative  feedback  can  be  combined  in  other 
ways  to  model  other  types  of  systems  as  well,  such  as  systems  involving  epidemics 
of  infectious  diseases  (Sterman,  2000).  Thus,  the  two  archetypes  can  have  different 
interactions  between  them  and  therefore  tell  different  stories. 

In  summary,  certain  configurations  of  elements  in  system  dynamics  modeling 
can  be  seen  as  representing  different  archetypes  that  can  have  different  types  of 
interactions  among  them.  In  so  doing,  the  archetypes  can  tell  a  wide  variety  of 
stories-as  in  traditional  storytelling-which  can  serve  to  help  the  reader  transfer 
meaning  and  inferences  from  the  simulation  (mental  simulation  in  storytelling; 
mathematical  simulation  in  system  dynamics)  to  situations  in  everyday  life.  Both 
storytelling  and  system  dynamics  modeling  involve  the  processes  of  simulation, 
analogical  reasoning,  and  pattern  recognition  which  involve  the  use  of  archetypes. 
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4.  USING  SCENARIOS  TO  ARCHIVE  EXPERIENCE 
AND  ORGANIZE  TRAINING 


INTRODUCTION 

Scenarios  are  widely  recognized  as  an  effective  way  to  engage  trainees  directly  in 
their  training;  this  interactivity  is  consistent  with  newer  educational  approaches 
that  promote  active  learning,  such  as  the  Adult  Learning  Model  (Dean,  1994; 
Prevou  &  Colorado,  2003)  and  constructivist  theories  of  instruction  (Bruner,  1 966, 
1996).  Coincident  with  these  instructional  methodologies  are  major  advances  in 
simulation  technology,  giving  rise  to  scenario-based  training  (SBT),  where  the 
development  and  delivery  of  compelling  scenarios  via  high  or  moderate  fidelity 
simulators  are  the  focal  points  of  the  instructional  process  (Salas,  Wilson,  Priest,  & 
Guthrie,  2006).  This  can  be  partially  contrasted  with  scenario-driven  training,  in 
which  scenarios  play  a  supporting  role  but  are  not  the  foundational  instructional 
unit  (Alexander,  2000). 

This  chapter  addresses  two  questions  concerning  the  use  of  scenarios  for  design 
and  delivery  of  SBT.  First,  how  can  we  optimize  scenario  construction  to  embody 
important  operational  experiences  that,  in  turn,  instill  the  essential  cognitive  and 
behavioral  skills,  knowledge,  and  attitudes  (KSAs)  the  trainee  is  to  acquire? 
Second,  how  can  we  use  scenarios  to  structure,  guide,  and  organize  training?  As 
a  backdrop  to  these  two  issues,  we  first  discuss  the  distinguishing  features  of 
scenarios,  their  advantages  and  disadvantages,  and  how  scenarios  have  evolved  to 
their  present  form.  Following  the  two  questions,  we  conclude  by  discussing  several 
trends  we  see  on  the  horizon  concerning  scenario  design,  development,  and 
implementation. 

In  discussing  the  chapter’s  titular  questions,  we  will  interleave  relevant  findings 
from  evaluations  of  some  half-dozen  SBT  projects  we  have  conducted  over  the 
past  few  years  (Spiker,  2006a).  Where  appropriate,  we  invoke  best  practices, 
borrowed  from  instructional  system  design  (ISD)  (Spiker,  2006b).  Sadly, 
development  and  delivery  of  SBT  is  at  best  only  lightly-principled  (Baker,  Kuang, 
Feinberg,  &  Radtke,  2004),  though  we  are  optimistic  that  this  state  of  affairs  is 
improving  (Ross,  Phillips,  Klein,  &  Cohn,  2005). 

SCENARIOS  AS  A  STORYTELLING  METHOD 

As  one  member  of  the  class  of  verbally-rich  storytelling  methods  that  comprise  this 
book  -  the  others  are  narratives,  problems,  and  cases  —  scenarios  do  indeed  tell  a 
story  to  trainees.  Yet  it  is  a  particular  type  of  story,  with  features  that  make 
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scenarios  particularly  well-suited  for  training.  Among  these  are  their  purposive 
nature  that  constrains  the  otherwise  free  play  that  tends  to  occur  in  simulators 
(Salas  et  al.,  2006),  along  with  associative  properties  that  make  them  personally 
relevant  to  the  trainee  (Baldwin  &  Madjuka,  1997).  Scenarios  share  a  number  of 
features  with  the  other  storytelling  instructional  methods.  These  include  an 
emphasis  on  grabbing  and  holding  the  student’s  attention,  promoting  active 
learning,  instilling  a  learner-centered  orientation,  and  a  desire  to  have  the  student 
internalize  the  experience  to  support  subsequent  transfer  to  the  post-training 
environment  (Fowlkes,  Dwyer,  Oser,  &  Salas,  1998). 

In  common  language  (Dictionary.com),  scenario  has  multiple  meanings,  such  as 
a  setting  of  story,  a  postulated  sequence  of  possible  events,  or  an  outline  of  the  plot 
or  action,  among  others.  As  applied  to  training,  scenario  usage  is  more  focused, 
where  it  encapsulates  a  blueprint  of  what  will  happen  that  defines  the  roles  of 
trainees  and  supporting  non-trainee  personnel;  thus  it  comes  to  dictate  the  content 
of  what  is  to  be  trained.  As  espoused  by  Salas  and  his  colleagues,  scenarios  in  a 
SBT  or  event-based  learning  environment  are  the  curriculum  (Oser,  Gualtieri, 
Canon-Bowers,  &  Salas,  1999;  Salas  &  Cannon-Bowers,  2001). 

ADVANTAGES  AND  POSSIBLE  PITFALLS  OF  SCENARIOS 

Scenarios  enjoy  a  number  of  advantages  as  a  framework  for  training  content. 
Besides  helping  to  establish  cognitive  realism  and  trainee  buy-in,  scenarios: 
provide  a  contextual  basis  for  trainees  to  more  readily  adopt  their  role  in  the  action; 
define  the  roles  for  non-trainees;  lay  out  the  sequence  of  action  that  will  unfold; 
dictate  the  specific  events  that  will  occur;  provide  a  timeline  for  those  events; 
and  facilitate  communication  among  members  of  the  training  team  (Spiker,  2006a; 
Van  Berio,  1998).  In  these  ways,  scenarios  help  prevent  chaos  in  the  training 
environment;  provide  an  antidote  to  unscripted  free  play  in  the  simulator;  make 
training  more  interesting,  believable  (i.e.,  more  face  validity),  personable,  and 
customizable;  and  make  training  reflect  the  motivational  goals  advocated  in  models 
of  adult  learning  (Wlodkowski,  1993). 

But  despite  these  advantages,  scenarios  do  not  guarantee  success,  as  there  are 
pitfalls  to  avoid.  For  example,  while  scenarios  are  the  essence  of  the  SBT  curriculum, 
they  are  not  a  substitute  for  training  objectives.  Within  ISD,  a  training  objective 
specifies  the  skill  to  be  acquired,  the  performance  standards  to  be  achieved,  and  the 
conditions  under  which  the  skill  is  to  be  executed  (Department  of  Defense,  2001). 
Unless  these  attributes  are  built  into  the  foundation  of  the  scenario,  the  trainer  cannot 
be  sure  that  a  given  scenario  will  accomplish  these  aims.  Using  a  scenario  to  guide 
training  also  does  not  ensure  that  criterion  levels  of  proficiency  have  been  achieved 
unless  explicit  goals  for  performance  were  set.  That  is,  completing  a  scenario  does 
not  guarantee  that  one  has  been  trained  unless  explicit  criteria  for  successful  training 
are  provided  (US  Air  Force,  2002).  Finally,  employment  of  scenarios  within  an  SBT 
context  does  not  ensure  that  a  trainee’s  acquired  skills  will  transfer  positively  to  the 
criterion  or  operational  environment;  some  type  of  test  for  the  presence  of  positive 
transfer  must  be  performed  to  establish  that  the  scenario  training  environment  is 
effective  (Schmidt  &  Bork,  1 992). 
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DISTINGUISHING  FEATURES  OF  SCENARIO 


Though  its  learning  goals  are  clearly  in  line  with  the  other  story-based  methods 
of  instruction,  scenarios  have  several  important  distinguishing  features.  One  of 
these  is  the  highly  structured  nature  of  scenarios,  where  well-designed  scenarios 
will  include  both  a  specification  of  the  stimulus  events  to  be  delivered  and  an 
enumeration  of  the  trainee  responses  that  can  be  expected  (Hooper  &  Hannafin, 
1991).  The  latter  is  especially  important  since  anticipating  trainee  responses 
gives  the  instructor  some  advance  idea  where  the  scenario  might  be  going,  a 
template  for  assessing  performance,  and  items  for  use  in  the  debrief  (Dismukes, 
Jobe,  &  McDonnell,  1997).  Structure  is  indeed  quite  important,  and  in  fact,  Bills 
(1997)  reported  that  having  a  structured  learning  environment  was  even  more 
important  than  interactivity  for  successful  performance  with  Internet-based 
instruction. 

Another  hallmark  of  scenarios  is  that  they  must  be  authored  by  someone,  as 
opposed  to  being  largely  or  wholly  borrowed  from  other  sources  the  way  that 
narratives,  cases,  and  —  to  a  lesser  extent  -  problems,  can  be.  Our  experience  has 
been  that  scenario  authoring  is  difficult  and  while  subject  matter  experts  (SMEs) 
have  the  technical  expertise  to  populate  a  scenario  with  challenging  events,  they 
still  need  guidance  in  laying  out  the  elements  of  a  scenario  in  a  way  that  promotes 
effective  learning  (Spiker,  Walls,  &  Karp,  2006). 

Another  distinguishing  feature  of  scenarios  is  their  inherent  dynamic  character, 
where  there  is  an  implicit  or  explicit  timeline  dictating  the  pace  and  sequence  of 
events  (Van  Berio,  1998).  The  temporal  placement  of  events  is  a  major  facet  of 
scenarios,  stemming  from  the  emphasis  of  SBT  on  KSA  practice  and  the 
concomitant  goal  of  efficiently  compressing  skill  acquisition.  The  need  for  learning 
efficiency  arises  because  high  fidelity  training  simulators  are  a  scarce  training 
resource,  and  there  is  an  urgent  requirement  to  maximize  the  trainees’  learning  rate 
to  free  up  the  device  for  the  next  trainee  (Salas  et  al.,  2006). 

Scenarios  are  often  designed  around  a  series  of  decision  points,  where  a  trainee’s 
particular  response  to  some  event  will  determine  the  branches  or  sequels  he/she 
receives  next  (Prince,  Oser,  Salas,  &  Woodruff,  1993).  Rich  scenarios  can  contain 
many  decision  points  and  multiple  paths,  but  some  means  must  be  invoked  to  keep 
the  number  of  choices  at  each  decision  point  to  an  acceptable  number,  else  trainee 
and  analyst  will  be  overwhelmed.  One  strategy  is  to  break  a  lengthy  scenario  into 
smaller  vignettes  or  frames,  where  each  frame  contains  a  single  decision  point 
(Pomerol,  2001).  At  the  end  of  the  frame,  the  instructor  has  the  option  of  resetting  the 
scenario  to  some  earlier  condition,  thereby  avoiding  a  combinatorial  explosion  of 
possible  branches.  Decomposing  mission  scenarios  into  smaller  frames  makes 
scenario  authoring  more  manageable  and,  importantly,  supports  modularization  of 
SBT  (Gillan,  2003). 

Another  feature  of  scenarios  is  the  need  for  associated  performance  measurement, 
either  through  automated  recording  or  live  observers,  and  feedback.  Regarding  the 
latter,  if  real-time  (i.e.,  during  the  training  session)  feedback  is  not  possible,  then 
delayed  feedback,  in  the  form  of  after  action  reviews  and  debriefs,  must  be  provided 
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and  incorporated  into  the  scenario  (Dismukes  et  al.,  1997).  The  scenario  is,  thus,  a 
very  dynamic  context  that  does  not  require  that  trainees  receive  an  extended  history 
before  beginning  skill  practice. 

Perhaps  most  importantly,  scenarios  are  authored  to  train  particular  KSAs,  so 
events  are  scripted  to  ensure  that  the  targeted  KSAs  are  required  (Dormann  & 
Frese,  1 994).  These  KS  A  triggers  are  thus  an  essential  part  of  the  scenario.  Figure  1 
illustrates  how  this  might  be  implemented.  This  example  is  taken  from  software 
developed  for  authoring  training  scenarios  for  use  in  Navy  and  commercial 
airline  flight  training  (Walls,  Spiker,  &  Hunt,  2006).  These  triggers  can  either 
occur  (a)  automatically,  based  on  time  into  the  mission,  phase  of  flight,  or  the 
trainee’s  response;  or  (b)  manually,  when  inserted  by  the  instructor.  The  event, 
Fuel  low  level  caution  light,  might  be  triggered  automatically,  when  the  aircraft 
reaches  a  certain  altitude  (5500’),  or  the  instructor  might  use  his/her  discretion  to 
generate  the  event  based  on  some  condition  designed  to  challenge  the  trainee. 
Because  operational  conditions  can  give  rise  to  any  number  of  triggers  for  a  given 
scenario  event,  an  authoring  shell  is  useful  for  capturing  important  triggers  and 
archiving  them  for  later  use.  Besides  the  challenge  factor,  if  the  events  and  their 
triggers  are  carefully  designed,  sequenced,  and  integrated  into  the  scenario,  there 
will  be  minimal  unproductive  time  for  the  trainee.  This  is  particularly  important  in 
full-mission  simulations,  where  considerable  time  is  needed  to  set  up  the  event 
(e.g.,  doing  extended,  unchallenging  flying  in  the  visual  database  in  order  to  reach 
a  threat  area),  and  for  team  training  simulation,  where  multiple  team  members  must 
be  managed  (Salas,  Bowers,  &  Rhodenizer,  1998). 
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Figure  1.  Automatic  and  manual  insertion  of  triggering  events  in  a  scenario. 

EVOLUTION  OF  SCENARIOS  IN  SBT 

Viewing  simulator  training  as  an  enterprise,  the  role  of  scenarios  and  the  story  they 
tell  have  evolved  through  three  distinct  phases,  depicted  in  Figure  2.  When  SBT 
was  first  introduced  to  military  and  industry,  scenarios  were  presented  in  the 
context  of  large,  geo-specific  databases  having  a  high  physical  fidelity  with  the 
criterion  environment.  The  scenarios  were  created  by  SMEs,  where  there  was  little 
capability  to  measure  any  aspect  of  trainee  performance  (Salas  et  al.,  2006), 
particularly  cognitive.  Early  instructor  operator  stations  (IOSs)  were  inflexible  and 
hard  to  use,  so  trainees  had  to  adapt  to  the  scenario  as  originally  designed  since 
instructors  were  unable  to  modify  scenario  elements  to  tailor  simulator  events  to 
the  trainees’  level  of  expertise  (Spiker,  Nullmeyer,  &  Tourville,  1998). 
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Figure  2.  Three  phases  of  scenario  evolution  within  scenario-based  training. 

Though  highly  detailed,  these  mission  scripts  contained  little  information  about 
how  to  instruct,  what  to  measure,  or  what  training  objectives  should  be 
accomplished.  Because  these  scenarios  were  costly  to  create  -  teams  of  SMEs  might 
labor  for  a  week  or  more  to  develop  one  -  and  even  harder  to  change,  the  same 
scenario  might  be  used  repeatedly.  In  extreme  cases,  the  same  training  scenario 
(i.e.,  same  threats,  emergency  events,  weather,  etc.)  would  be  used  for  years,  so  that 
trainees  receiving  refresher  training  might  see  the  same  scenario  multiple  times, 
clearly  degrading  its  training  effectiveness  (Nullmeyer  &  Spiker,  2000).  As  might  be 
expected,  trainee  response  to  these  repeated  scenarios  was  not  high  either,  and 
declined  further  with  each  successive  repetition  of  the  same  scenario. 

During  the  second  phase,  which  we  have  been  in  for  some  time,  scenarios 
became  smaller  and  more  regional  in  scope,  more  cognitive  in  orientation,  more 
likely  to  be  created  by  a  project  team,  and  could  include  stopping  points  for 
measuring  performance.  With  improvements  in  IOS  technology,  it  has  become 
easier  to  change  the  contextual  features  of  the  scenario  database  -  threats,  weather, 
visibility  conditions,  time  of  day  -  so  the  difficulty  level  of  the  session  could  be 
altered  to  match  the  trainees’  level  of  expertise.  The  story  has  also  become  more 
interesting,  where  specific  events  could  be  inserted  to  tax  particular  cognitive 
skills,  such  as  decision  making,  situation  awareness,  and  problem  solving  (Salas 
et  al.,  2006).  Importantly,  there  ensued  greater  ability  to  practice  the  science  of 
training ,  through  repeated  practice  opportunities,  richer  feedback,  and  more 
focused  debriefs  (Salas  &  Cannon-Bowers,  2001). 

In  the  third  phase,  whose  onset  is  imminent,  scenarios  will  become  even  smaller 
in  scale  and  scope,  emphasizing  meta-cognitive  as  opposed  to  purely  cognitive  or 
psychomotor  KSAs.  Scenarios  will  be  used  to  structure  training  in  which  events 
will  be  presented  in  an  adaptive  fashion,  with  difficulty  levels  manipulated  in  real¬ 
time  to  match  the  trainees’  present  level  of  skill  and  knowledge.  Performance 
measurement  capabilities  will  be  embedded  into  the  scenario,  and  with  the  advent 
of  scenario  authoring  tools,  virtually  anyone  will  be  able  to  create  or  tailor  scenarios. 
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as  less  subject  matter  expertise  and  training  analytic  skill  will  be  required. 
Scenarios  will  be  created  to  emphasize  a  particular  meta-cognitive  skill  (e.g., 
critical  thinking,  adaptive  decision-making,  cultural  awareness),  where  high  levels 
of  transfer  of  training  will  be  expected  (Gillan,  2003).  With  widespread  use  of 
generic  databases  not  tied  to  a  particular  locale,  scenario  authors  will  have 
expanded  freedom  to  tell  a  story  with  broader  appeal  and  even  greater  utility 
(Lussier  &  Shadrick,  2004).  This,  in  turn,  will  open  up  the  possibilities  of  new 
technologies,  pedagogical  constructs,  and  paradigms,  such  as  serious  games, 
intelligent  agents,  and  adaptive  systems  (Salas  et  al.,  2006).  These  issues  are 
discussed  further  at  the  conclusion  of  the  chapter. 

INCORPORATING  USER  EXPERIENCES  INTO  SCENARIOS 

Unlike  cases  and  problems,  the  story  told  by  a  scenario  does  not  have  to  be  factual, 
though  most  often  it  will  have  been  based  on  the  operational  experiences  of 
instructors  or  training  analysts.  The  scenario  might  also  be  based  on  a  recorded 
event,  such  as  a  mishap  or  near  miss,  but  more  often  the  inspiration  comes  from 
war  stories  that  were  experienced  or  reported,  with  embellishments  to  target 
specific  KSAs  for  training.  An  efficient  way  to  collect  many  such  experiences  at 
one  time  is  to  hold  critical  incident  (Cl)  workshops,  where  experts  convene 
individually  or  as  a  group  to  relate  one  or  more  encounters  they  had  on  the  topics 
of  interest  (Hanson,  Hedge,  Logan,  Bruskiewicz,  Borman,  &  Siem,  1995).  In  the 
past  few  years,  the  Cl  technique  has  been  transformed  by  Klein  and  his  associates 
into  the  Cognitive  Decision  Making  (CDM)  method,  in  which  a  series  of  ever- 
deepening  probe  questions  are  asked  on  a  given  incident  (Seamster,  Redding,  & 
Kaempf,  2000).  The  CDM  yields  a  richer  tapestry  of  cognitive  detail,  shedding 
particular  light  on  decisions,  potential  sources  of  errors,  response  alternatives,  and 
overall  subjective  impressions  of  the  participating  SMEs. 

Stimulating  SME  recall  through  workshops  or  CDM  protocols  is  just  the 
beginning  step  in  constructing  a  face-valid  scenario.  In  addition,  a  structured 
method  is  needed  to  categorize  that  information  in  a  format  that  can  be  picked  up 
by  scenario  designers  who  may  not  be  an  SME.  We  have  found  that  using  a 
framework  like  that  shown  in  Figure  3  is  a  good  way  to  standardize  scenario 
development  and  ensure  that  all  requisite  story  information  is  in  place  for  the 
designer  (Spiker  et  al.,  2006a).  As  shown  in  the  figure,  we  can  decompose  a 
scenario  into  five  main  elements:  a  (1)  synopsis  or  storyline  that  summarizes  the 
main  actions  that  occur;  (2)  the  non-trainee  entities  or  players  who  must  be 
represented  to  make  the  story  realistic  and  contextually  meaningful;  (3)  the 
individual  training  events  that  are  specified,  selected,  and  inserted  as  the  basic  unit 
of  action;  (4)  supporting  materials,  physical  artifacts,  and  other  content  that  must 
be  supplied  by  the  SBT  system;  and  (5)  a  timeline  that  serves  as  a  scaffold  on 
which  to  place  events  (Spiker,  Holder,  Walls,  Campsey,  &  Bruce,  2007).  In  this 
illustration,  we  have  represented  the  training  scenario  for  a  simulated  commercial 
airline  flight  by  specifying  the  (1)  takeoff  and  departure  airports,  (2)  need  for  three 
role-playing  entities,  including  air  traffic  control;  (3)  five  different  events  that  will 
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P* a«ers 

Events 

Content 
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Basic  daytime  flight  from  Dulles  to 
Nashville.  During  start,  several 
interrupted  start  problems  are 
encountered  and  discussed.  Normal 
SID  using  autopilot.  Practice  area 
entry  for  steep  turns  and  stalls  In 
practice  area,  low  hydraulic  pressure 
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Pack  to  Dudes,  using  autopilot  Full 
stop  landing  is  completed 
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Turns 
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HYD 

Approach 

ILS 

Figure  3.  Example  of  a  five-part  scenario  framework. 

be  introduced,  such  as  requirement  for  steep  turns;  (4)  different  pilotage  charts  that 
must  be  available  during  the  flight;  and  (5)  points  in  the  flight  profile  where  the 
various  events  are  to  occur. 

Over  the  course  of  developing  SBT  systems  in  several  domains,  we  have 
learned  that  formal  methods  are  needed;  not  only  to  stimulate  the  recall  of 
operational  experience,  but  to  capture  these  insights  in  order  to  populate  our 
training  regime  with  interesting  and  compelling  scenarios.  This  can  be  done  in  a 
variety  of  ways,  including  worksheets,  checklists,  and  formal  scripts.  Each  format 
has  its  advantages,  such  as  ease  of  use,  graphical  convenience,  and  completeness, 
among  others.  Importantly,  our  evaluation  results  have  shown  that  trainees  and 
trainers  alike  react  positively  to  having  consistently  designed  scenarios  in  the 
training  curriculum,  where  acceptance  has  been  high  for  commercial  airline  pilots 
(Spiker  et  al.,  2006a),  Air  Force  mission  planners  (Spiker,  Walls,  &  Holder,  2006), 
Navy  pilots  (Walls  et  al.,  2006),  and  Army  staff  officers  (Fischer,  Spiker,  Harris,  & 
McPeters,  2006;  Spiker  et  al.,  2007). 

Interpreting  the  results  of  field  evaluations  of  SBT  is  notoriously  difficult  since  the 
no-scenario  control  condition  is  typically  absent,  making  simultaneous  comparisons 
impossible.  Instead,  we  rely  on  a  combination  of  measures  taken  from  archival  records 
(e.g.,  grade  sheets),  instructor  interviews,  and  trainee  survey  responses,  where  baseline 
data  are  compared  with  corresponding  measures  collected  after  SBT  has  been 
implemented.  In  this  way,  we  infer  the  benefits  that  might  have  resulted  from  SBT  in 
general,  and  structured  methods  of  archiving  experience  to  support  scenarios,  in 
particular.  The  difficulties  associated  with  forming  definitive  conclusions  under  quasi- 
experimental  conditions  have  been  well-documented  (Salas,  Burke,  Bowers,  & 
Wilson,  2004).  Nevertheless,  positive  reports,  even  if  based  on  non-experimental, 
qualitative  accounts,  can  at  least  give  us  a  sign  that  we  are  on  the  right  track. 

For  example,  one  can  use  less  experienced  training  analysts  to  create  scenarios 
when  there  is  a  structured  framework  as  discussed  above  (Spiker  et  al.,  2006a),  since 
they  can  rely  less  on  their  personal  experience  and  more  on  the  archival  accounts. 
An  archiving  framework  also  allows  the  scenario  to  be  parceled  out  to  multiple 
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designers.  The  framework  can  be  further  enhanced  by  adding  a  sixth  column,  in 
which  the  training  events  are  categorized  by  type  of  problem.  Categorization 
becomes  particularly  important  when  a  number  of  training  events  have  been 
identified  in  the  domain,  where  the  optimal  categorization  scheme  will  depend  on  the 
job  domain.  In  the  area  of  flight  training,  a  particularly  useful  scheme  has  been  to 
classify  events  as  falling  into  one  of  four  problem  areas:  operational,  environmental, 
equipment,  and  human  factors  (Spiker,  2006a). 

Classifying  events  by  problem  type  permits  a  logical  ordering  based  on  severity, 
frequency,  and  types  of  problems  the  student  is  expected  to  overcome.  Using  this 
scheme,  analysts  can  design  a  scenario  whose  content  difficulty  is  optimal  for  the 
student’s  current  level  of  proficiency.  Besides  yielding  an  optimal  level  of  difficulty, 
drawing  content  events  from  a  categorization  scheme  can  help  create  scenarios  of 
comparable  difficulty,  though  populated  with  different  training  events.  This  keeps 
the  scenario  regime  fresh  and  unpredictable  (Dubois  &  Gillan,  2000).  Using  the 
four-problem  area  scheme  mentioned  in  the  paragraph  above,  one  could  produce  a 
scenario  in  which  the  trainees  must  takeoff  with  an  aircraft  whose  gross  weight 
exceeds  recommended  limits  (operational  problem).  They  might  then  experience, 
in  succession,  inadvertent  instrument  meteorological  conditions  (environmental 
problem),  loss  of  radar  altimeter  (equipment  problem),  and  an  incapacitated  crew 
member  (human  factors  problem). 

Ease  of  scenario  construction  can  be  a  double-edged  sword,  however.  While 
archiving  frameworks  can  help  populate  a  scenario  with  more  detail,  this  can 
overwhelm  trainees  if  left  unchecked  (Fischer  et  al.,  2006).  This  problem  can  occur 
in  several  ways.  On  the  one  hand,  ease  of  scenario  development  encourages  the 
designer  to  inject  more  story  lead-ins  and  backdrops  for  a  given  set  of  training 
events.  In  conducting  formative  evaluations  of  our  training  program  for  critical 
thinking,  we  discovered  that  users  thought  that  the  scenario  vignettes  varied  too 
often  with  problem  sets.  Users  instead  preferred  having  a  common  back  story 
structure  (e.g.,  a  single  operational  order  that  they  would  then  review)  as  the  basis 
for  subsequent  exercises  (Fischer  et  al.,  2006). 

The  other  pitfall  with  easier  development  is  that  a  greater  amount  of  scenario 
material  itself  can  be  generated.  This  is  particularly  true  when  training  cognitive 
KSAs,  where  the  most  frequent  criticism  of  scenario  background  information  is 
that  they  entail  too  much  reading  (Spiker  et  al.,  2007).  The  multi-media  literature 
has  consistently  shown  that  instructional  materials  need  to  minimize  reading  as 
much  as  possible  (Mayer,  2001),  so  the  experience-archiving  system  that  one 
adopts  should  reflect  this  need.  For  example,  one  can  use  scenario  templates 
where  storylines  are  word-limited  and  supporting  graphics  are  always  required. 
Interestingly,  all  of  the  SBT  evaluations  we  have  conducted  in  recent  years  have 
found  that  a  highly  valued  aspect  of  training  is  the  ability  to  convene  as  a  group, 
such  as  for  roundtable  discussions.  While  this  activity  may  be  off  script  for  our 
individual  training  scenarios,  it  is  well-received  by  participants  and  serves 
important  team  training  functions.  One  should  also  consider  having  such  stopping 
points  in  a  scenario,  where  possible,  to  allow  for  face  to  face  discussions,  thus 
producing  a  blended  training  experience  (Spiker  et  al.,  2007). 
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One  of  the  most  important  reasons  for  having  structured  archiving  tools  for 
scenario  development  is  the  assurance  it  provides  for  technical  accuracy  and 
operational  relevance  of  the  scenario.  These  features  have  consistently  been  found 
to  dictate  user  acceptance,  even  more  so  than  ease  of  use  and  other  interface 
features  (Fischer  et  al.,  2006).  To  achieve  this  realism,  we  have  found  that  it  is 
important  to  work  with  SMEs  of  all  types,  where  it  is  essential  that  we  have  tools 
that  capture  their  expertise.  The  framework,  checklist,  worksheet,  and  event 
specification  frames  depicted  in  Figures  3—4  help  ensure  that  this  expertise  is 
properly  archived. 

Since  trainees  and  instructors  do  not  want  to  be  beta-testers,  internal  review  of 
scenario  quality  is  important  as  well.  Figure  4  depicts  examples  of  screen  displays 
that  inform  scenario  designers  when  they  have  violated  pre-established  rules 
concerning  missing  parameters  (e.g.,  leaving  out  aircraft  fuel  amount),  initial 
conditions,  and  logical  conflicts  (Walls  et  al.,  2006).  Prior  to  generating  the 
completed  scenario  as  output,  these  quality  control  indices  are  invoked  and  give 
designers  clues  on  how  to  improve  their  scenario.  Employment  of  automatic 
quality  checking  both  increases  scenario  design  efficiency  and  reduces  the 
amount  of  in-simulator  fixes  that  have  to  be  employed  once  the  scenario  has  been 
installed. 


■  Quality  AsMiraiii  v  Review  HU 

Good  To  Co  -'*PH  A  ttvntran  R*4uu«i 

SimDATT  HAS  identified  missing  parameters*  in  your  session. 

If  you  intend  for  a  parameter  to  be  "Not  Selected"  or  blank  then  no 
action  is  required,  otherwise  provide  an  appropriate  entry. 

0  Missing  Parameter  Check 

mm 

0  Conflicting  ConCiDons  Check  | 

mm 

*  SimDATT  checks  a  N-notr  if  important  parameter*  to  see  ■(  they 
are  mnsmg.  It  does  nof  rfceek  parameter 

0  Session  PitfaBs  Check 

mm 

Pi-’il  ihs  Seawi  Review  CtMGkfact 
and  complete  4  offline 

No  Initial  Fuel  level  information  has  been  provided  for  this  session. 

0  Session  Review  Checklist 

PRINT  ' 

Fuel  Level:  |  K 

SimDATT  HAS  identified  session  pit  faffs*  in  your  session. 

If  you  intend  for  a  situation  to  exist  "as  is"  then  no  action  is 
required,  otherwise  make  an  appropriate  change. 

•  SimDATT  only  checks  for  a  limited  number  of  common  pitfalls. 

mm 

The  following  flight  segments  contain  more  than  three  malfunctions/complications.  In  most 
cases,  this  leads  to  unrealistic  training. 

ENROUTE 


Return  to  the  Build  Profile  tab  to  revise  this  scenario 
Go  To  Build  Profile 


Figure  4.  Example  of  a  scenario  quality  control  (QC)  checking  function. 
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Finally,  our  evaluations  of  SBT  have  indicated  that,  through  interviews  and 
formal  surveys,  instructors  have  little  time  to  develop  and  refine  scenarios  (Walls 
et  al.,  2006).  In  fact,  in  many  cases  development  of  new  training  scenarios  amounts 
to  an  added  duty.  Given  the  benefits  to  a  training  program  that  accrue  from  creating 
rich  scenarios,  it  is  imperative  that  structured  archiving  tools  be  available  to  help 
instructors  and  staff  SMEs  record  their  experiences  as  efficiently  as  possible.  The 
multiple  job  demands  on  front-line  instructors  require  that  this  expertise  be 
extracted  quickly,  so  other,  less  technically-versed  individuals  can  interact  with  the 
material  to  package  it  into  a  more  complete  scenario  format.  In  other  words,  having 
scenario  archiving  and  design  tools  that  separate  the  KSA-content  from  pedagogical 
aspects  of  instruction  is  an  important  ingredient  to  a  successful  SBT  program 
(Spiker  et  al.,  2006a). 


USING  SCENARIOS  TO  ORGANIZE  TRAINING 

Scenarios  not  only  offer  a  useful  means  for  recording  and  classifying  training 
events,  they  also  provide  an  effective  way  to  organize  the  delivery  of  training. 
Since  scenarios  are  the  primary  curriculum  element  in  SBT,  they  must  be 
represented  in  ways  that  they  can  be  accessed  quickly  and  inserted  into  the 
training  flow  at  the  appropriate  time.  Because  ISD  was  developed  prior  to 
widespread  use  of  SBT,  there  is  still  no  official  guidance  for  how  scenarios  are  to 
be  integrated  into  a  curriculum  (Spiker,  2006a).  Below,  we  discuss  five 
techniques  we  believe  support  delivery  of  SBT  lessons  and  exercises  in  an 
organized  fashion. 

A  simple  but  effective  way  to  organize  SBT  is  to  house  the  scenarios  in  a 
library.  Key  to  this  approach  is  having  a  viewing  area  that  contains  a  thumbnail 
sketch  of  each  scenario,  so  users  may  rapidly  search  the  library  for  scenarios  of 
immediate  interest.  This  concept  is  illustrated  in  Figure  5,  where  the  upper  part  lists 
the  scenarios  by  name  while  the  lower  part  shows  a  thumbnail  sketch  for  the 
highlighted  scenario.  Moving  the  cursor  in  the  library  window  will  cause  the 
thumbnail  sketch  in  the  lower  window  to  display  the  summary  information  of  the 
corresponding  scenario.  In  this  example,  our  summary  contains  the  major  defining 
variables  for  the  flight  training  domain,  including  synopsis,  events,  difficulty  level, 
and  estimated  duration.  Depending  on  the  application,  it  may  be  desirable  to  have 
other  scenario  features  depicted  there,  such  as  objectives  or  KSAs,  among  others. 
Importantly,  a  library  organization  helps  time-pressured  training  analysts  quickly 
find  scenarios  appropriate  for  upcoming  SBT  sessions.  Exposing  the  training  staff 
to  the  distinguishing  features  of  scenarios  in  the  summary  window  increases  their 
core  knowledge,  which  in  turn,  will  help  them  modify  scenarios  to  meet  their 
immediate  training  needs  (Prince  et  al.,  1993).  In  this  regard,  our  evaluations  have 
shown  that  SBT-users  are  more  likely  to  create  and  contribute  new  scenarios  when 
they  have  ready  access  to  a  central  scenario  from  which  modular  offshoots  can  be 
produced  (Walls  et  al.,  2006). 
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Figure  5.  Example  of  a  scenario  library  and  viewing  portal. 

An  effective  long-term  strategy  for  organizing  SBT  is  to  link  each  scenario  to 
a  set  of  underlying  training  objectives,  allowing  the  objectives  themselves  to  be 
the  organizing  theme.  Unfortunately,  SBT  has  a  history  of  having  open-ended 
simulator  sessions  without  any  underlying  objectives  (Salas  et  al.,  2006).  With  that 
deficit  in  mind,  we  created  a  scenario  development  system,  SimDATT,  to  help 
Navy  training  analysts  develop  cognitively-challenging  flight  profile  scenarios  for 
fixed  wing  aircraft  (Walls  et  al.,  2006).  Each  scenario  was  created  to  satisfy  one  or 
more  explicit  training  objectives. 

In  a  formative  evaluation  of  the  system,  feedback  from  our  test  subjects  revealed 
that  it  was  not  enough  to  specify  training  objectives,  as  these  can  seem  somewhat 
ponderous  and  obtuse  to  students  who  are  in  the  midst  of  the  curriculum  (Van  Berio, 
2005).  As  well,  the  objectives  should  be  linked  to  the  student’s  specific  learning  goals. 
This  linkage  is  illustrated  in  Figure  6,  where  each  training  objective  (on  the  left)  is 
translated  into  concrete  learning  experiences  the  trainee  is  expected  to  leave  with. 
Thus,  specifying  the  student  learning  goals,  in  addition  to  training  objectives,  becomes 
a  more  powerful  way  to  organize  training  since  it  allows  instructors  to  describe  the 
upcoming  scenario  to  the  student  in  terms  that  are  directly  relevant  to  them. 
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Figure  6.  Linking  scenario  objectives  to  trainee  learning  goals. 

A  third  organizational  technique  is  to  use  a  staged  progression  of  scenarios, 
where  scenarios  are  ordered  according  to  basic  principles  of  ISD  (Swezey  & 
Llaneras,  1997).  Among  these  are  the  complexity  and  difficulty  of  the  KSAs  to  be 
trained,  where  it  is  advisable  to  present  the  simpler  and  easier  scenarios  first, 
followed  by  scenarios  whose  events  are  progressively  more  complex  and  difficult. 
Difficulty  can  be  manipulated  by  having  setup  conditions  that  approximate  the  real 
world  (e.g.,  weather,  visibility,  winds,  air  traffic,  aircraft  configuration),  and  with 
training  events  that  overlap  in  time  and  occur  in  higher  densities  (Greitzer,  Pond,  & 
Jannotta,  2004).  Besides  organizing  scenarios  by  complexity  and  difficulty,  it  is 
important  to  have  the  presentation  sequence  ensure  that  trainees  experience  marked 
variability  in  the  conditions,  as  this  is  a  proven  way  to  increase  transfer  of  training 
to  the  criterion  environment  (Schmidt  &  Bjork,  1992).  Likely  reasons  for  such 
improved  transfer  include  keeping  trainee  workloads  to  a  manageable  level, 
avoiding  intense  frustration  periods  that  can  demoralize  students,  adhering  to  a 
building  block  approach  (crawl  —  walk  -  run),  as  well  as  allowing  for  a  more 
natural,  incremental  acquisition  of  complex  skilled  behavior  (Spiker,  2006a). 

A  fourth  method  for  organizing  SBT  delivery  is  to  give  the  instructor  flexibility 
over  how  events  are  injected  into  the  scenario.  This  will  usually  require  that  the 
IOS  be  programmed  to  support  both  automatic  and  manual  delivery  of  events.  The 
former  will  be  pre-determined  based  on  certain  trigger  events  in  the  environment, 
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such  as  at  a  specific  time  or  location  in  the  session.  However,  manual  delivery  of 
events  can  be  useful  since  the  trainee’s  responses  to  prior  events  cannot  always  be 
predicted;  hence,  the  scenario  might  proceed  down  a  path  for  which  pre-programmed 
events  are  no  longer  appropriate.  In  these  instances,  allowing  the  instructor  to 
inject  response-contingent  events  is  effective  for  maintaining  an  appropriate 
difficulty  level.  In  implementing  this  dual  method  of  control,  the  list  of  possible 
events  contains  a  mix  of  scripted  (planned)  and  unscripted  events,  where  the  latter 
lets  the  instructor  tweak  the  scenario  to  fit  the  trainee’s  proficiency  level.  This 
duality  also  makes  scenario  control  adhere  to  principles  of  deliberate  practice 
(Ericsson,  Krampe,  &  Tesche-Romer,  1993),  where  feedback  and  repetitive  practice 
are  explicitly  designed  to  improve  performance  on  a  targeted  KSA.  Recall  that  a 
key  part  of  a  scenario’s  story  is  its  anticipation  of  possible  responses,  correct  and 
incorrect,  that  trainees  will  make  to  each  event.  Incorporating  response-contingent 
control  into  the  IOS  programming  of  training  events  is  a  good  way  to  handle 
unanticipated  trainee  responses  (Spiker  et  al.,  2007). 

Finally,  a  key  organizational  capability  entails  progressive  or  telescopic 
content  development.  With  this  method,  the  problem  space  of  the  scenario  is 
designed  using  progressive  increase  in  detail.  In  the  use  of  our  SimDATT  tool  for 
pilot  training,  the  authoring  sequence  is:  (1)  desired  overall  KSA  behaviors/concepts, 
(2)  instructional  objectives  to  be  achieved,  (3)  the  basic  training  flight  profile  to 
be  flown,  (4)  the  training  scenario  events  to  be  accomplished,  and  (5)  a  detailed 
script  for  instructors  and  any  role-playing  entities  (Spiker,  2006a).  For  example, 
in  constructing  a  lesson  plan  scenario  for  a  CRJ  200  flight  training  device,  we 
started  with  an  overall  concept  of  inserting  a  low  hydraulic  pressure  indication 
into  the  flight  profile.  The  instructional  designer  then  took  this  notion  and 
progressively  increased  its  detail  and  placed  it  into  the  scenario.  This  culminated 
in  Step  5,  as  a  script  for  the  instructor,  to  be  spoken  right  after  the  event,  role- 
playing  air  traffic  control  to  distract  the  trainee  and  increase  scenario  difficulty 
(Spiker,  et  al.,  2006a). 

Telescoping  helps  the  instructor  keep  the  big  picture  in  mind  before  drilling 
down  to  the  specific  details  that  will  actually  be  programmed  in  the  simulator.  Not 
only  is  the  general-to-specific  technique  a  proven  design  method,  it  is  an  effective 
approach  for  any  planning  activity  (Van  Berio,  1998).  In  our  SBT  evaluations, 
telescoping  content  down  to  the  kernel  KSAs  was  viewed  as  an  essential  capability 
of  any  scenario  authoring  system  (Spiker  et  al.,  2006a). 

CONCLUSIONS  AND  FUTURE  DIRECTIONS 

Like  its  storytelling  cousins,  scenarios  provide  an  effective  way  to  engage  trainees 
in  the  instructional  process  by  stressing  interactive  learning  and  critical  thinking. 
Scenarios  have  additional  features  that  make  for  a  good  training  framework. 
Among  these  are  an  implicit  or  explicit  timeline  of  action,  an  event-based  focus 
that  elicits  behaviors  underlying  the  to-be-acquired  KSAs,  training  objectives,  and 
individual  trainee  goals.  When  scenario  design  includes  a  specification  of  potential 
trainee  responses  to  each  event  and  a  verbal/behavioral  script  for  the  instructor,  it 
then  creates  a  solid  foundation  on  which  SBT  can  be  based. 
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This  chapter  discussed  two  aspects  of  SBT:  (1)  using  scenarios  to  capture  and 
archive  experience  and  (2)  organizing  the  delivery  of  SBT  in  the  context  of  these 
scenarios.  Regarding  the  former,  scenarios  capture  user  experiences,  perhaps 
embellishing  them,  to  make  a  compelling,  engaging  story  that  draws  the  trainee  in. 
Because  scenario  construction  is  difficult,  even  for  technical  experts,  tools  and 
techniques  are  needed  to  help  SMEs  and  instructors  fold  operational  experience 
into  scenario  design.  Besides  critical  incident  workshops,  other  effective  methods 
for  stimulating  and  archiving  experience  include  structured  frameworks,  checklists, 
worksheets,  and  even  formal  scripts.  Evaluations  of  SBT  reveal  that  capturing 
operational  details  into  scenarios  increases  user  acceptance.  But  this  must  be 
balanced  against  giving  too  much  detail  that  can  overwhelm  inexperienced 
trainees.  Having  an  automated  method  of  computing  scenario  quality,  presently  in 
its  experimental  stages,  is  likely  to  be  an  effective  method  in  the  future  to  ensure 
that  experience  has  been  correctly  archived  within  the  scenario. 

With  regard  to  organizing  SBT  delivery,  five  techniques  are  particularly  helpful. 
A  simple  method  involves  placing  scenarios  into  a  library,  where  highlighted 
scenario  entries  have  an  accompanying  summary  window  of  key  information  items 
for  rapid  selection.  Other  organizing  techniques  include  linking  scenarios  to 
training  objectives  and  associated  trainee  learning  goals,  ordering  scenarios  on  the 
basis  of  complexity  or  ease  of  learning,  providing  instructors  with  the  means  to 
inject  training  events  either  automatically  or  manually,  and  using  a  progressive  or 
telescoping  method  of  content  development.  With  this  latter  technique,  scenarios 
are  developed  in  stages,  starting  with  underlying  KSAs,  and  followed  in  turn  by 
instructional  objectives,  mission  profile  or  event  timeline,  the  events  themselves, 
and  a  detailed  script  for  the  instructor  and  all  role-playing  entities. 

While  advances  have  been  made  in  our  understanding  of  scenario  design  and  SBT 
delivery,  more  work  is  clearly  needed.  One  area  in  need  of  more  study  concerns  the 
evaluation  of  SBT  effectiveness.  As  convincingly  described  in  Salas  et  al.  2001, 
evaluations  of  training  systems  are  difficult  to  conduct  since  the  no-scenario  control 
group  is  rarely  fielded.  Assessments  are  thus  limited  to  demonstrating  user  acceptance, 
with  scant  data  collected  on  degree  of  learning,  transfer,  or  organizational  impact 
(Kirkpatrick,  1987).  Innovative  techniques  are  needed  to  collect  data  from  users 
possessing  a  corporate  memory  to  provide  valid  ratings  following  SBT  implemen¬ 
tation  of  what  has  improved,  what  has  declined,  and  what  has  not  changed  at  all. 
Besides  baseline/post-implementation  differences,  experimental  contrasts  of  SBT  and 
true  non-scenario  control  conditions  need  to  be  conducted  where  practical. 

A  promising  development  in  scenario  design  is  the  increasing  use  of  powerful 
scenario  authoring  tools  and  wizards.  Such  tools  will  guide  the  novice  analyst  through 
all  the  steps  needed  to  archive  experiences,  create  storylines,  define  events,  and  piece 
them  together  on  a  timeline  in  pedagogically  effective  ways.  Coupled  with  embedded 
performance  measurement  and  help  features,  wizards  can  support  both  adaptive 
training,  customizing  training  events  in  real-time  to  the  trainee’s  current  level  of 
proficiency,  and  just-in-time  training,  while  adhering  to  a  standardized  set  of  KSAs 
and  training  events.  Authoring  tools  will  also  permit  less  experienced  training  analysts 
to  be  productive  participants  in  training  design  and  delivery. 


50 


SCENARIOS  IN  EXPERIENCE  AND  TRAINING 


Peering  out  further  into  the  future,  there  are  other  new  capabilities  that  could 
promote  SBT.  For  example,  one  can  use  serious  games  -  games  intended  to  provide 
education  or  instruction  -  as  a  way  to  package  scenarios  in  a  competitive  but 
entertaining  environment  (Mautone,  Spiker,  &  Dick,  2007).  Another  trend  involves 
the  use  of  wikis  to  create  collaborative  knowledge  environments  that  reach  back  to 
operational  units  and  incorporate  their  feedback  in  drafting  topical  scenarios. 

Finally,  scenario  development  and  delivery  are  part  of  a  global  transformation  in 
training  in  which  training  is  really  just  one  aspect  of  an  individual’s  personal  and 
lifelong  professional  development.  Thus,  advances  are  not  confined  to  formal 
training  episodes  in  the  classroom  or  simulator  (Day  &  Halpin,  2003).  This  implies 
that  student  motivation  must  be  emphasized  to  ensure  that  performance  continues 
to  improve  beyond  that  required  during  formal  training.  Distance  learning,  where 
scenarios  can  be  accessed  and  experienced  on  the  trainee’s  home  computer  on  their 
own  time,  must  be  brought  under  the  scenario  rubric  to  ensure  that  principles  of 
effective  scenario  design  and  implementation,  as  discussed  above,  are  adhered  to  in 
the  ubiquitous  Internet  environment.  Since  trainees’  internal  transformations  are 
mobile  and  long-lasting,  we  will  need  well-designed  scenario  tools  and  delivery 
practices  to  help  ensure  that  this  transformation  proceeds  in  a  positive  direction.1 

NOTES 

1  Portions  of  the  research  presented  in  this  chapter  were  funded  by  contracts  from  the  Naval  Air 
Systems  Command  (N61339-02-C-0135),  Army  Research  Institute  (DASW01-04-C-008,  W74V8H- 
04-C-0007),  and  Air  Force  Research  Laboratory  (F336 1 5-03-C-63 1 9).  The  author  wishes  to  thank 
the  following  individuals  who  served  as  contract  monitors  on  those  respective  contracts:  Dr.  Melissa 
Walwanis-Nelson,  Dr.  Michelle  Zbylut,  Sharon  Riedel,  and  Dr.  Robert  Nullmeyer. 
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